Categorical Cross-Entropy

If you'd like more details about fully connected layers in Keras, check out the documentation for the Dense layer. You can change the way the weights are initialized through supplying values for the kernel_initializer and bias_initializer parameters. Note that the default values are 'glorot_uniform', and 'zeros', respectively. You can read more about how each of these initializers work in the corresponding Keras documentation.
There are many different loss functions in Keras. For this lesson, we will only use categorical_crossentropy.
Check out the list of available optimizers in Keras. The optimizer is specified when you compile the model (in Step 7 of the notebook).
- 'sgd' : SGD
- 'rmsprop' : RMSprop
- 'adagrad' : Adagrad
- 'adadelta' : Adadelta
- 'adam' : Adam
- 'adamax' : Adamax
- 'nadam' : Nadam
- 'tfoptimizer' : TFOptimizer